Upstream tracking #165
Draft
Upstream tracking #165
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously builtins.readDir would return an empty attribute set instead of barfing on non-existent paths. This is a regression from 2.32 for impure eval. (cherry picked from commit 4ab2cda)
…tenance [Backport 2.33-maintenance] libutil/union-source-accessor: Barf on non-existent directories
Make path inputs lazy
Previously, calling queryValidPaths() with a large number (e.g. 100K) of store paths failed because Nix immediately creates a `TransferItem` for each .narinfo, which is then registered as a handle with curl. However curl appears to scale poorly internally: even though only a few downloads are actually started (up to the connections/streams limits), it spends a lot of CPU time dealing with the inactive handles. So the curl thread is sitting at 100% CPU, the active downloads stall and time out, and everything grind to a halt. So now we limit the number of curl handles to http-connections * 5. With this, fetching 100K .narinfo files from localhost succeeds in ~15 seconds.
There can be a long time between the creation of `TransferItem` and the start of the curl download, which can lead to misleading download durations and progress bar status. So now we create the `Activity` and update `startTime` when curl actually starts the download.
…hang-when-querying-many-narinfos-in-nix-3121 Limit the number of active curl handles
…detnix-version-string-not-displayed-in-nix-repl nix repl: Show Determinate version
…wdown packaging: fix build with lowdown
This is especially important when using a chroot store and `path:...` refers to the host store.
Tagging release 2.33.1
Path fetcher: Handle invalid paths
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Not intended to be merged directly. This PR is a convenience to show the diff between upstream Nix and Determinate Nix (the
mainbranch).Continuation of #4.